Refinement of the Child Amblyopia Treatment Questionnaire (CAT-QoL) using Rasch analysis

Aims or Purpose: The Child Amblyopia Treatment Questionnaire (CAT-QoL) was developed using a "bottom-up" methodological approach. Interviews with children with amblyopia identified items (questions) and response levels to be tested in a draft questionnaire consisting of 11 items (sad, feeling on face, hurt, doing schoolwork, cross, how other children treat you, doing things, worried, upset with family, playing with friends, happy). This study describes the refinement of the descriptive system for the CAT-QoL instrument using the application of Rasch analysis. METHODS: A multi-centre pilot study was conducted, and data collected from 342 participants. Participants were asked to self-complete the appropriate treatment version of the CAT-QoL questionnaire socio-demographic and clinical data were collected by the clinician using a standardised proforma. A "measure" of child's health was obtained from the parent by asking how they would rate their child's health over the previous week. Rasch analysis techniques were applied to refine the questionnaire. Rasch was used to examine response categories and collapse item response levels, identify poorly performing items, and explore local dependency of items. RESULTS: A total of 331 subjects were included in the study sample, however only 315 were accepted into the RUMM program as a number of subjects had missing questions responses on the CAT-QoL. RUMM also excluded a further 41 subjects as these demonstrated extreme responses. Disordered response categories were found for each item, requiring adjacent response levels to be combined. This was applied to all items, and the model fit was re-examined. Two items were found to have poor fit (cross and happy) and were removed from the measure and the model fit was re-examined. No statistically significant differential item functioning (DIF) was found for any item, using person factors of age, sex or general health. Two items showed some dependency (worried and upset with family), and the poorer fitting item was subsequently removed (upset with family). This resulted in a refined CAT-QoL instrument that consists of 8-items, each with three-level response scales. CONCLUSION: The refined CAT-QoL instrument includes the following items: sad, feeling on face, hurt, doing work at school, how other children treat you, doing things, worried and playing with friends. The CAT-QoL can be Rasch scored, with a range of 0-16 where a greater value indicates a worse quality of life (or greater impact of treatment on the individual). The CAT-QoL may be useful in determining how amblyopia treatment affects children, and offers an alternative to generic patient reported outcome measures

Conceptualisation, development and validation of T-QoL© (Teenagers' Quality of Life): a patient-focused measure to assess quality of life of adolescents with skin diseases

Aim To develop and validate a dermatology-specific quality of life (QoL) instrument for adolescents with skin diseases. Methods Qualitative semi-structured interviews were conducted with adolescents with skin disease to gain in-depth understanding of how skin diseases affect their QoL. A prototype instrument based on the themes identified from content analysis of interviews was tested in several stages, using Classical Test Theory (CTT) and Item Response Theory (IRT) models to develop this new tool and conduct its psychometric evaluation. Results Thirty-three QoL issues were identified from semi-structured interviews with 50 adolescents. A questionnaire based on items derived from content analysis of interviews was subjected to Rasch analysis: factor analysis identified three domains, therefore not supporting the validity of T-QoL as a unidimensional measure. Psychometric evaluation of the final 18-item questionnaire was carried out in a cohort of 203 adolescents. Convergent validity was demonstrated by significant correlation with Skindex-Teen and CDLQI or DLQI. The T-QoL showed excellent internal consistency reliability: Cronbach's α=0.89 for total scale score and 0.85, 0.60, and 0.74 respectively for domains 1, 2 and 3. Test-retest reliability was high in stable subjects. T-QoL showed sensitivity to change in two sub-groups of patients who indicated change in their self-assessed disease severity. Conclusion Built on rich qualitative data from patients, the T-QoL is a simple and valid tool to quantify the impact of skin disease on adolescents’ QoL; it could be used as an outcome measure in both clinical practice and clinical research

Reliability and responsiveness of measures of pain in people with osteoarthritis of the knee: a psychometric evaluation

PURPOSE: To examine the fit between data from the Short Form McGill Pain Questionnaire (SF-MPQ-2) and the Rasch model, and to explore the reliability and internal responsiveness of measures of pain in people with knee osteoarthritis. METHODS: Participants with knee osteoarthritis completed the SF-MPQ-2, Intermittent and Constant Osteoarthritis Pain questionnaire (ICOAP) and painDETECT. Participants were sent the same questionnaires 3 and 6 months later. RESULTS: Fit to the Rasch model was not achieved for the SF-MPQ-2 Total scale. The Continuous subscale yielded adequate fit statistics after splitting item 10 on uniform DIF for gender, and removing item 9. The Intermittent subscale fit the Rasch model after rescoring items. The Neuropathic subscale had relatively good fit to the model. Test-retest reliability was satisfactory for most scales using both original and Rasch scoring ranging from fair to substantial. Effect sizes ranged from 0.13 to 1.79 indicating good internal responsiveness for most scales. CONCLUSIONS: These findings support the use of ICOAP subscales as reliable and responsive measure of pain in people with knee osteoarthritis. The MPQ-SF-2 subscales found to be acceptable alternatives. Implications for Rehabilitation The McGill Pain Questionnaire short version 2 is not a unidimensional scale in people with knee osteoarthritis, whereas three of the subscales are unidimensional. The McGill Pain Questionnaire short version 2 Affective subscale does not have good measurement properties for people with knee osteoarthritis. The McGill Pain Questionnaire short version 2 and the Intermittent and Constant Osteoarthritis Pain scales can be used to assess change over time. The painDETECT performs better as a screening measure than as an outcome measure

KIDMAP, a web based system for gathering patients' feedback on their doctors

<p>Abstract</p> <p>Background</p> <p>The gathering of feedback on doctors from patients after consultations is an important part of patient involvement and participation. This study first assesses the 23-item Patient Feedback Questionnaire (PFQ) designed by the Picker Institute, Europe, to determine whether these items form a single latent trait. Then, an Internet module with visual representation is developed to gather patient views about their doctors; this program then distributes the individualized results by email.</p> <p>Methods</p> <p>A total of 450 patients were randomly recruited from a 1300-bed-size medical center in Taiwan. The Rasch rating scale model was used to examine the data-fit. Differential item functioning (DIF) analysis was conducted to verify construct equivalence across the groups. An Internet module with visual representation was developed to provide doctors with the patient's online feedback.</p> <p>Results</p> <p>Twenty-one of the 23 items met the model's expectation, namely that they constitute a single construct. The test reliability was 0.94. DIF was found between ages and different kinds of disease, but not between genders and education levels. The visual approach of the KIDMAP module on the WWW seemed to be an effective approach to the assessment of patient feedback in a clinical setting.</p> <p>Conclusion</p> <p>The revised 21-item PFQ measures a single construct. Our work supports the hypothesis that the revised PFQ online version is both valid and reliable, and that the KIDMAP module is good at its designated task. Further research is needed to confirm data congruence for patients with chronic diseases.</p

Reliability and Validity of the Telephone-Based eHealth Literacy Scale Among Older Adults: Cross-Sectional Survey

Background: Only a handful of studies have examined reliability and validity evidence of scores produced by the 8-item eHealth literacy Scale (eHEALS) among older adults. Older adults are generally more comfortable responding to survey items when asked by a real person rather than by completing self-administered paper-and-pencil or online questionnaires. However, no studies have explored the psychometrics of this scale when administered to older adults over the telephone. Objective: The objective of our study was to examine the reliability and internal structure of eHEALS data collected from older adults aged 50 years or older responding to items over the telephone. Methods: Respondents (N=283) completed eHEALS as part of a cross-sectional landline telephone survey. Exploratory structural equation modeling (E-SEM) analyses examined model fit of eHEALS scores with 1-, 2-, and 3-factor structures. Subsequent analyses based on the partial credit model explored the internal structure of eHEALS data. Results: Compared with 1- and 2-factor models, the 3-factor eHEALS structure showed the best global E-SEM model fit indices (root mean square error of approximation=.07; comparative fit index=1.0; Tucker-Lewis index=1.0). Nonetheless, the 3 factors were highly correlated (r range .36 to .65). Item analyses revealed that eHEALS items 2 through 5 were overfit to a minor degree (mean square infit/outfit values <1.0; t statistics less than –2.0), but the internal structure of Likert scale response options functioned as expected. Overfitting eHEALS items (2-5) displayed a similar degree of information for respondents at similar points on the latent continuum. Test information curves suggested that eHEALS may capture more information about older adults at the higher end of the latent continuum (ie, those with high eHealth literacy) than at the lower end of the continuum (ie, those with low eHealth literacy). Item reliability (value=.92) and item separation (value=11.31) estimates indicated that eHEALS responses were reliable and stable. Conclusions: Results support administering eHEALS over the telephone when surveying older adults regarding their use of the Internet for health information. eHEALS scores best captured 3 factors (or subscales) to measure eHealth literacy in older adults; however, statistically significant correlations between these 3 factors suggest an overarching unidimensional structure with 3 underlying dimensions. As older adults continue to use the Internet more frequently to find and evaluate health information, it will be important to consider modifying the original eHEALS to adequately measure societal shifts in online health information seeking among aging populations.Open Access Fundin

Rasch analysis of the Patient and Observer Scar Assessment Scale (POSAS) in burn scars

The Patient and Observer Scar Assessment Scale (POSAS) is a questionnaire that was developed to assess scar quality. It consists of two separate six-item scales (Observer Scale and Patient Scale), both of which are scored on a 10-point rating scale. After many years of experience with this scale in burn scar assessment, it is appropriate to examine its psychometric properties using Rasch analysis. Cross-sectional data collection from seven clinical trials resulted in a data set of 1,629 observer scores and 1,427 patient scores of burn scars. We examined the person-item map, item fit statistics, reliability, response category ordering, and dimensionality of the POSAS. The POSAS showed an adequate fit to the Rasch model, except for the item surface area. Person reliability of the Observer Scale and Patient Scale was 0.82 and 0.77, respectively. Dimensionality analysis revealed that the unexplained variance by the first contrast of both scales was 1.7 units. Spearman correlation between the Observer Scale Rasch measure and the overall opinion of the clinician was 0.75. The Rasch model demonstrated that the POSAS is a reliable and valid scale that measures the single-construct scar qualit

Rasch analysis of the Psychiatric Out-Patient Experiences Questionnaire (POPEQ)

<p>Abstract</p> <p>Background</p> <p>The Psychiatric Out-Patient Experiences Questionnaire (POPEQ) is an 11-item core measure of psychiatric out-patients experiences of the perceived outcome of the treatment, the quality of interaction with the clinician, and the quality of information provision. The POPEQ was found to have evidence for reliability and validity following the application of classical test theory but has not previously been assessed by Rasch analysis.</p> <p>Methods</p> <p>Two national postal surveys of psychiatric outpatients took place in Norway in 2004 and 2007. The performance of the POPEQ, including item functioning and differential item functioning, was assessed by Rasch analysis. Principal component analysis of item residuals was used to assess the presence of subdimensions.</p> <p>Results</p> <p>6,677 (43.3%) and 11,085 (35.2%) psychiatric out patients responded to the questionnaire in 2004 and 2007, respectively. All items in the scale were retained after the Rasch analysis. The resulting scale had reasonably good fit to the Rasch model. The items performed the same for the two survey years and there was no differential item functioning relating to patient characteristics. Principal component analysis of the residuals confirmed that the measure to a high degree is unidimensional. However, the data also reflects three potential subscales, each relating to one of the three included aspects of health care.</p> <p>Conclusions</p> <p>The POPEQ had excellent psychometric properties and Rasch analysis further supported the construct validity of the scale by also identifying the three subdimensions originally included as components in the instrument development. The 11-item instrument is recommended in future research on psychiatric out-patient experiences. Future development may lead to the construction of more precise measures of the three subdomains that the POPEQ is based on.</p

Assessment of examiner leniency and stringency ('hawk-dove effect') in the MRCP(UK) clinical examination (PACES) using multi-facet Rasch modelling

BACKGROUND: A potential problem of clinical examinations is known as the hawk-dove problem, some examiners being more stringent and requiring a higher performance than other examiners who are more lenient. Although the problem has been known qualitatively for at least a century, we know of no previous statistical estimation of the size of the effect in a large-scale, high-stakes examination. Here we use FACETS to carry out a multi-facet Rasch modelling of the paired judgements made by examiners in the clinical examination (PACES) of MRCP(UK), where identical candidates were assessed in identical situations, allowing calculation of examiner stringency. METHODS: Data were analysed from the first nine diets of PACES, which were taken between June 2001 and March 2004 by 10,145 candidates. Each candidate was assessed by two examiners on each of seven separate tasks. with the candidates assessed by a total of 1,259 examiners, resulting in a total of 142,030 marks. Examiner demographics were described in terms of age, sex, ethnicity, and total number of candidates examined. RESULTS: FACETS suggested that about 87% of main effect variance was due to candidate differences, 1% due to station differences, and 12% due to differences between examiners in leniency-stringency. Multiple regression suggested that greater examiner stringency was associated with greater examiner experience and being from an ethnic minority. Male and female examiners showed no overall difference in stringency. Examination scores were adjusted for examiner stringency and it was shown that for the present pass mark, the outcome for 95.9% of candidates would be unchanged using adjusted marks, whereas 2.6% of candidates would have passed, even though they had failed on the basis of raw marks, and 1.5% of candidates would have failed, despite passing on the basis of raw marks. CONCLUSION: Examiners do differ in their leniency or stringency, and the effect can be estimated using Rasch modelling. The reasons for differences are not clear, but there are some demographic correlates, and the effects appear to be reliable across time. Account can be taken of differences, either by adjusting marks or, perhaps more effectively and more justifiably, by pairing high and low stringency examiners, so that raw marks can be used in the determination of pass and fail